chore: Optimize store #3061
chore: Optimize store #3061
Conversation
|
This PR may contain changes to database schema of one of the drivers. If you are introducing any changes to the schema, make sure the upgrade from the latest release to this change passes without any errors/issues. Please make sure the label |
|
You can find the image built from this PR at Built from 19af1f1 |
Ivansete-status
force-pushed
the
optimize-store-queries
branch
from
6eeac6f
to
2cea2aa
Compare
Ivansete-status
force-pushed
the
optimize-store-queries
branch
2 times, most recently
from
c6f3249
to
d3a33da
Compare
Ivansete-status
force-pushed
the
optimize-store-queries
branch
from
d3a33da
to
ae5cd5f
Compare
Ivansete-status
changed the title
Ivansete-status
requested review from
richard-ramos,
darshankabariya and
gabrielmer
There was a problem hiding this comment.
LGTM, thanks so much!
Quick question - does doing
await sleepAsync(0.milliseconds)
pollute the pending Futures?
My understanding is that it gives control back to the event loop and the sleep's future is cleaned promptly, there shouldn't be more than one instance of the sleepAsync(0.milliseconds) in the pending Futures at once. But maybe I'm wrong
Co-authored-by: gabrielmer <[email protected]>
Yes I understood it the same way but after some analysis, I got the impression that it doesn't work exactly that way |
| pool.conns[i].busy = false | ||
|
|
||
| const SlowQueryThresholdInNanoSeconds = 2_000_000_000 | ||
| const SlowQueryThresholdInNanoSeconds = 1_000_000_000 |
There was a problem hiding this comment.
Just a hint, you may like it.
const SlowQueryThresholdInNanoSeconds = 1.seconds
... and use it if needed like...
SlowQueryThresholdInNanoSeconds.nanos()
I feel it more readable instead of counting the zeros :-)
WDYT would it work?
There was a problem hiding this comment.
It's much better that way thanks! I've applied the suggestion in 54e635d
Description
We have three big-time consumption components in
store:This PR enhances the point 2, because we've been applying an incorrect
asyncapproach, which was to wait for a certain state with the following technique:or
Doing that has a very big impact on performance because we were adding a massive amount of future objects to the async queue. @arnetheduck - correct me if that statement is not accurate.
Therefore, in this PR we are starting to apply a better
asyncapproach to properly coordinate tasks.Aside from that, we are simplifying
pgasyncpool.nimand suggesting a better wrapper for theDbConnobject.We are also optimizing the following query so that we use the
messages_lookuptable instead:SELECT timestamp FROM messages WHERE messageHash = $1